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Instructional Conversations and Literature Loos 


Program description This WWC report examines the effect of Instructional Conversa- 
tions and Literature Logs used in combination. The goal of Instruc- 
tional Conversations is to help English language learners develop 
reading comprehension ability along with English language profi- 
ciency. Instructional Conversations are small-group discussions. 
Acting as facilitators, teachers engage English language learners 


in discussions about stories, key concepts, and related personal 
experiences, which allow them to appreciate and build on each 
others’ experiences, knowledge, and understanding. Literature 
Logs require English language learners to write in a log in response 
to writing prompts or questions related to sections of stories. 

These responses are then shared in small groups or with a partner. 


Research Two studies of Instructional Conversations and Literature 
Logs met the What Works Clearinghouse (WWC) evidence 
standards with reservations. 1 The two studies included over 
200 Hispanic English language learners from grades 2-5. The 
two studies reviewed for this report assess program impacts 


in two different contexts; one focuses on the short-term (use 
of the intervention over a few days) and the other focuses on 
the long-term (use of the intervention over a few years) with 
the intervention delivered as key components in a broader 
language arts program. 



Effectiveness Instructional Conversations and Literature Logs was found to have potentially positive effects on reading achievement and English 
language development. 



English language 

Reading achievement Mathematics achievement development 



Rating of effectiveness Potentially positive effects Not reported 

Improvement index 2 Average: +29 percentile points Not reported 

Range: +24 to +33 percentile 
points 



Potentially positive effects 
Average: +23 percentile points 
Range: +21 to +24 percentile 
points 



1. The evidence presented in this report is based on available research. Findings and conclusions may change as new research becomes available. 

2. These numbers show the average and range of improvement indices for all findings across the studies. 
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Developer and contact 

The Instructional Conversations and Literature Logs intervention 
was developed by William Saunders and Claude Goldenberg 
and is available from the publisher, the Center for Research on 
Education, Diversity & Excellence (CREDE). Address: Graduate 
School of Education, 1640 Tolman Hall, University of California, 
Berkeley, CA 94720-1670. Email: crede@berkeley.edu. Web: 
http://crede.berkelev.edu . Telephone: (510) 643-9024. 

Scope of use 

Information is not available on the number or demographics of 
students, schools, or districts using this intervention. 

Teaching 

This WWC report focuses on a specific approach to combining 
the Literature Logs and Instructional Conversations used in the 
Saunders and Goldenberg (1999) and Saunders (1999) 3 studies. In 
the Saunders and Goldenberg (1999) study, the instructional com- 
ponents were used on alternating days with the same story. For the 
Literature Log component, teachers assigned entries for different 
segments of readings. English language learners were then asked 



to write about personal experiences relevant to a character in the 
story, provide a detailed description of an event that occurred in 
the story, and evaluate a theme from the story. They wrote in their 
logs independently and then participated in a discussion led by 
the teacher comparing their own personal experiences to those of 
the character in the story. For Instructional Conversation lessons, 
the teacher facilitated discussions with English language learners 
in small groups about the content of the story. They participated 
in discussions for approximately 45 minutes per week. The 
discussions provide an opportunity for teachers to assess story 
comprehension and assist in broadening English language learn- 
ers’ understanding of the story content and themes. 

The two studies reviewed in this report used the intervention in 
different ways. In the Saunders and Goldenberg (1999) study, 
Instructional Conversations and Literature Logs was used as a 
short-term intervention and delivered within four days. In the Saun- 
ders (1999) study, the intervention was delivered as a key compo- 
nent of a broader language arts program delivered from grades 2-5. 

Cost 

No information is currently available on the cost of the intervention. 



Two studies reviewed by the WWC investigated the effects of 
Instructional Conversations and Literature Logs. One study 
(Saunders & Goldenberg, 1999) was a randomized controlled trial 
that met WWC evidence standards with reservations due to dif- 
ferences between groups in the amount of instruction provided. 4 
The other study (Saunders, 1999) was a quasi-experimental 
design that met WWC evidence standards with reservations. 



Saunders and Goldenberg’s (1999) short-term study matched 
58 students from two fourth grade and three fifth grade 
classrooms on language and reading proficiency and randomly 
assigned them to either the intervention or comparison group 
within each classroom. 5 Thirty-two of these students were 
English language learners and are the focus of this intervention 
report. The intervention group was compared to the comparison 



3. In addition to Instructional Conversations and Literature Logs, Saunders (1999) included 10 common instructional components in the experimental 
group and employed them over a five-year period. Readers may want to view the results of this study and Saunders and Goldenberg (1999) separately in 
order to judge the effectiveness of the Instructional Conversations and Literature Logs alone versus in combination with a variety of other more common 
instructional components. 

4. The study reported a confound between the intervention and the amount of instructional time because students in the intervention condition had more 
instructional time than those in the comparison group. This makes it difficult to ascertain whether the observed group differences resulted from the dif- 
ference in the amount of instruction or type of instruction. 

5. Saunders and Goldenberg (1999) included two additional study groups ( Instructional Conversations only and Literature Logs only). The results pertaining 
to these groups were not reviewed in this intervention report. 
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Research (continued) 



group on multiple reading achievement and English language 
development measures. 

Saunders (1999) conducted a longitudinal study of the effects 
of a transitional bilingual program on Hispanic English language 
learners. The program featured Instructional Conversations 
and Literature Logs as two key components integrated with a 
reading language arts curriculum, and was provided to English 
language learners from grades 2-5. Ninety English language 
learners from five program schools and 90 English language 
learners from five matched neighboring schools were randomly 
selected from the pool of eligible students 6 to participate in 



the study at the beginning of the second grade. From the 125 
English language learners who remained in the study through 
the end of fifth grade, 84 were randomly selected and matched 
within subgroups 7 based on first grade Spanish reading and 
language scores to form the analytic sample. The analysis 
compared the 42 English language learners in the intervention 
group and the 42 English language learners in the comparison 
group on multiple reading achievement and English language 
development measures. Instruction and assessment for these 
fifth grade English language learners were in English and the 
focus of this report is for that grade only. 



Effectiveness Findings 

The WWC review of English language learner interventions 
addresses student outcomes in three domains: reading 
achievement, mathematics achievement, and English language 
development. 

Reading achievement. Saunders and Goldenberg (1999) 
found that Instructional Conversations and Literature Logs had 
statistically significant positive effects on both measures of fac- 
tual comprehension and interpretive comprehension. The WWC 
was only able to confirm the statistical significance of the factual 
comprehension measure. Saunders (1999) reported a statisti- 
cally significant difference favoring the fifth grade intervention 
group on performance assessment. The WWC confirmed the 
statistical significance of this finding. In both studies, the inter- 
vention had statistically significant positive effects on reading 
achievement according to WWC criteria. 

English language development. Saunders (1999) reported 
a statistically significant difference favoring the fifth grade 



intervention group on English language development. The WWC, 
however, was not able to confirm the statistical significance of 
this finding. In this study, the effect of Instructional Conversa- 
tions and Literature Logs on English language development was 
substantively important. 

Rating of effectiveness 

The WWC rates the effects of an intervention in a given outcome 
domain as: positive, potentially positive, mixed, no discernible 
effects, potentially negative, or negative. The rating of effective- 
ness takes into account four factors: the quality of the research 
design, the statistical significance of the findings (as calculated 
by the WWC 8 ), the size of the difference between participants 
in the intervention and the comparison conditions, and the con- 
sistency in findings across studies (see the WWC Intervention 
Rating Scheme) . 



6. All students in the analysis sample were English language learners. 

7. Subgroups were formed based on whether the student qualified and began transitional language arts in fourth or fifth grade. This report does not sepa- 
rate findings by subgroup and instead focuses on findings based on the overall sample. 

8. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within class- 
rooms or schools and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch . See the Technical Details of WWC-Conducted 
Computations for the formulas the WWC used to calculate the statistical significance. In the case of Instructional Conversations and Literature Logs, 
corrections for multiple comparisons and clustering were needed. 
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The WWC found 
Instructional Conversations 
and Literature Logs to have 
potentially positive effects 
for reading achievement 

The WWC found Instructional 
Conversations and Literature 
Logs to have potentially 
positive effects for English 
language development 



References 



Improvement index 

The WWC computes an improvement index for each individual 
finding. In addition, within each outcome domain, the WWC 
computes an average improvement index for each study and 
an average improvement index across studies (see Technica 
Details of WWC-Conducted Computations) . The improvement 
index represents the difference between the percentile rank 
of the average student in the intervention condition versus 
the percentile rank of the average student in the comparison 
condition. Unlike the rating of effectiveness, the improvement 
index is entirely based on the size of the effect, regardless of 
the statistical significance of the effect, the study design, or the 
analysis. The improvement index can take on values between 
-50 and +50, with positive numbers denoting favorable results. 
The average improvement index for reading achievement is +29 
percentile points across the two studies, with a range of +24 to 
+33 percentile points across findings. The improvement index for 



English language development is +23 with a range of +21 to +24 
percentile points across findings. 

Summary 

The WWC reviewed two studies on Instructional Conversations and 
Literature Logs. One of these studies was a randomized controlled 
trial that met WWC evidence standards with reservations due to 
differences between groups in the amount of instruction given. 

This study focused on the short-term impacts of the intervention. 
The other study was a quasi-experimental design that met WWC 
evidence standards with reservations. This study focused on 
the long-term impacts where the intervention was administered 
within a language arts program. Based on these two studies, the 
WWC found the intervention to have potentially positive effects on 
reading achievement and potentially positive effects on English 
language development. The evidence presented in this report is 
limited and may change as new research emerges. 



Met WWC evidence standards with reservations 
Saunders, W. M. (1999). Improving literacy achievement for 
English learners in transitional bilingual programs. Educational 
Research and Evaluation, 5(4), 345-381. 



Saunders, W. M., & Goldenberg, C. (1999). Effects of instruc- 
tional conversations and literature logs on limited- and 
fluent-English-proficient students’ story comprehension and 
thematic understanding. Elementary School Journal, 99(4), 
277-301. 



For more information about specific studies and WWC calculations, please see the WWC Instructional 
Conversations and Literature Loos Technical Appendices . 
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Appendix A1.1 


Study characteristics: Saunders & Goldenberg, 1999 (randomized controlled trial with reservations) 


Characteristic 


Description 


Study citation 


Saunders, W. M., & Goldenberg, C. (1999). Effects of instructional conversations and literature logs on limited- and fluent-English-proficient students' story comprehension 
and thematic understanding. Elementary School Journal, 99(4), 279-301. 


Participants 


All 138 English language learners enrolled in three fifth-grade and two fourth-grade classrooms in one school participated in this study. They were matched by language 
proficiency (limited or fluent) and teachers’ rating of reading skills, and then randomly assigned to four study conditions within each classroom. Twenty-two English language 
learners were excluded from the final analysis (3 special education students, 4 students enrolled just prior to the study, 12 students not present for some of the activities, and 
3 students randomly excluded to provide for a balanced design). The remaining 116 participants were evenly distributed among the four groups (29 per group). Only 32 of 
these participants in two study groups are of interest for this intervention report: 16 English language learners in the comparison condition and 16 English language learners in 
the Instructional Conversations and Literature Logs (intervention) condition. 1 These students participated in the study with students fluent in English. 


Setting 


The study took place in a K-5 elementary school located in an urban area. The majority of students at the school were Hispanic (82%), had limited English proficiency 
(69%), and qualified for the free or reduced-price lunch program (62%). More than three quarters of the fourth-grade students were performing below grade level in reading, 
language, and math. The school ranked among the lowest 20% of schools in the district. Schoolwide efforts were underway at the time of the study to improve bilingual 
programs, English language development program, language arts instruction, and overall academic infrastructure at the school. 


Intervention 


The study lasted for a period of 10-15 days, including pre- and post-intervention activities. The intervention was implemented over four days. On the first day, English 
language learners in the Literature Log only group and the Instructional Conversations and Literature Logs group received instruction on Literature Logs in two consecutive 
45-minute lessons. On the second day, English language learners in the Instructional Conversation only group and the Instructional Conversations and Literature Logs group 
received two 45-minute lessons on Instructional Conversations. The same procedures were followed on the third and fourth days, with the order of the lessons reversed (that 
is, Instructional Conversations lessons on day three and Literature Logs lessons on day four). English language learners in each group also spent at least 45 minutes on creat- 
ing an illustration and caption summarizing their interpretation of the story “Louella’s Song.” 


Comparison 


English language learners in the comparison group participated in reading and writing activities related to social studies either independently or with a teaching assistant. 

They also devoted at least 45 minutes to creating an illustration and caption summarizing their interpretation of the story “Louella’s Song.” Saunders and Goldenberg (1999) 
reported that English language learners in the comparison group did not receive as much direct instructional time from teachers as those in the Instructional Conversations and 
Literature Logs group. 2 


Primary outcomes 
and measurement 


The effects of the intervention were assessed using several measures, including factual comprehension, interpretive comprehension, theme-explanation essay, and theme- 
exemplification essay. Although theme-explanation and theme-exemplification essays were measures used in the study, outcomes were not reported in the WWC report 
because the data reported in the study were the percentage of English language learners whose essays received a high score. Therefore, the measures did not meet WWC 
standards (see Appendix A2.1 for more detailed descriptions of outcome measures). 


Teacher training 


The five teacher participants in the study were members of the research and development team implementing the school’s language arts model in Spanish, transition, and 
mainstream English language arts classrooms. The team was led by two instructional advisors who were able to co-teach and provide assistance in the classrooms on a daily 
basis. The team met on a bi-monthly basis to study instructional components, view videotapes and live demonstrations, plan instructional units, and evaluate English language 
learners’ work. All study conditions (including the control) were carried out in each classroom by each of the five teachers. 



1. The other two study groups ( Instructional Conversations only and Literature Logs only) in Saunders and Goldenberg (1999) were not of interest to this intervention report. 

2. The different amounts of teacher-led instruction in the intervention and comparison conditions might have confounded the effects of the intervention; therefore this study was downgraded to 

meeting WWC evidence standards with reservations. 
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Appendix A1.2 Study characteristics: Saunders, 1999 (quasi-experimental design) 



Characteristic 


Description 


Study citation 


Saunders, W. M. (1999). Improving literacy achievement for English learners in transitional bilingual programs. Educational Research and Evaluation, 5(4), 345-381 . 


Participants 


Eligible participants for the study were students who enrolled in their respective schools from first grade through fifth grade, considered Spanish their primary home language, 
had limited English proficiency at the time of enrollment, and were in a Spanish transitional bilingual program. Among the pool of eligible English language learners, 180 
were randomly selected at the beginning of second grade from five intervention schools and five neighboring schools, then matched on academic achievement, percentage of 
students with limited English proficiency, percentage of students in the free or reduced-priced lunch program, school enrollment, and ethnic composition. Of the 180 English 
language learners, 125 remained at their respective school through fifth grade, from which 42 intervention-group English language learners and 42 comparison-group English 
language learners matched on first grade Spanish reading and language and transition status were selected for data analysis. 


Setting 


The study took place in a school district in southern California. The average enrollment for the 10 participating schools was 880. Over 90% of the students enrolled in these 
schools were Latino or Hispanic, 74% were limited English proficient, and 95% qualified for free or reduced-price lunches. 


Intervention 


English language learners in the intervention group participated in a transitional bilingual program, which covered several transitional phases as they progressed from receiving 
instruction in their primary language to instruction in English. English language learners were in the Pre-Transition phase in the second and third grades, in Transition 1 in 
the fourth grade, and in Transition II in the fifth grade. Transition II is the only phase during which instruction was dedicated completely to reading and writing in English, and 
English language learners were assessed with English language measures. Transition II services were provided only in grade 4 or 5, depending on whether a student was ready 
for English-only instruction. The earlier transition phases involved academic support and assessments in both English and Spanish. This transition program incorporated 12 
instructional components, including Instructional Conversions and Literature Logs. Instructional Conversations involved ongoing discussions, while Literature Logs (and writing 
projects) entailed writing activities. Teachers assisted English language learners in understanding how the experiences of a character in a story relate to their own experiences. 
Writing assignments were intended to encourage English language learners to think about and express ideas, as well as extract meaning from the stories. The Instructional 
Conversations were intended to provide a forum for sharing ideas and a more complex understanding of story themes. English language learners studied literature during each 
phase, practiced using reading comprehension strategies, and read assigned books independently. The assignments were based on each English language learner’s reading 
level. Additionally, for 45 minutes per day during the Pre-Transition phase, teachers provided instruction using the English Language Development through Literature program. 
Instruction was provided in small groups comprised of English language learners with similar levels of English language proficiency. Three or more times per week, teachers 
read aloud for approximately 20 minutes. English language learners also chose a book or story to read independently each day and completed short assignments for the ones 
that they found most engaging. Additionally, they wrote in journals during the beginning stages of English writing and received written responses from the teacher. Instructional 
Conversations and Literature Logs were two of the 1 2 components of the language arts program that was implemented. 


Comparison 


English language learners in the comparison group met the same criteria as those in the intervention group. English language learners in the comparison group participated in 
the district's typical bilingual education program. In the primary grades, for approximately 20-30 minutes each day, English language learners received instruction in English 
language development. Instruction for the rest of the day was in Spanish. When they demonstrated proficiency in Spanish reading and writing, and basic proficiency in relation 
to oral English skills (the end of second grade through the beginning of third grade), they continued Spanish language arts, and began receiving reading and writing instruction 
in English (transition stage). After approximately 3-6 months, English language learners began a mainstream English program. The comparison group did not receive the 
transition program that incorporated 12 instructional components, including Instructional Conversations and Literature Logs, which were taught to English language learners in 
the intervention group. 


Primary outcomes 
and measurement 


English language learners’ reading and English language development were assessed using the Comprehensive Test of Basic Skills (CTBS) and the Criteria for Addition of 
Reading in English (CARE). Reading outcomes were also assessed using performance assessments. Because most English language learners in grades 1-4 took the CTBS 
and performance assessment in Spanish, findings from these grades are not included in this report. Only findings for fifth-graders, who took the tests in English, are included 
in this report (see Appendix A2.1 and A2.2 for more detailed descriptions of outcome measures). 


Teacher training 


Saunders (1999) noted that the teachers involved in the study worked with project staff for the full five years during which it was implemented. No further information about 
teacher training was reported. 
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Appendix A2.1 Outcome measures in the reading achievement domain 



Outcome measure 


Description 


Comprehensive Test 
of Basic Skills (CTBS), 
Reading Subtest 

Performance assessments: 
Reading achievement 


The CTBS is a standardized achievement test used to assess English language proficiency (as cited in Saunders, 1999). It includes subtests of reading and language. Perfor- 
mance on the CTBS was reported in terms of Norm Curve Equivalent (NCE) scores. 

The author developed this assessment, which consists of six tasks: narrative comprehension (measures story comprehension), informational comprehension (measures 
how well English language learners understand informational text), writing communication (measures the lucidity of the information that English language learners provide to 
peers about story content), writing conventions (measures spelling, punctuation, capitalization, and usage), dictation (measures spelling, punctuation, and capitalization), and 
independent reading and library use, in which English language learners indicate the books and stories that they have chosen to read, and items borrowed from the library (as 
cited in Saunders, 1999). The assessments were administered in a small-group setting and scored by trained project staff and teachers under the direction of the author. The 
inter-rater reliability was 61 % based on exact agreement and 97% based on adjacent agreement (agreement within one point with a five-point scale). 


Factual comprehension 


This measures required English language learners to recall factual details about a story. The measure was made up of 10 items (as cited in Saunders & Goldenberg, 1999). 


Interpretive comprehension 


This measure included five questions related to the interpretation of events that occurred in a story (as cited in Saunders & Goldenberg, 1999). 



Appendix A2.2 Outcome measures in the English language development domain 



Outcome measure 


Description 


Comprehensive Test 
of Basic Skills (CTBS), 
Language Subtest 

Performance assessments: 
Writing achievement 


The CTBS is a standardized achievement test used to assess English language proficiency (as cited in Saunders, 1999). It includes subtests of reading and language. Perfor- 
mance on the CTBS was reported in terms of Norm Curve Equivalent (NCE) scores. 

The author developed this assessment, which consists of six tasks: narrative comprehension (measures story comprehension), informational comprehension (measures 
how well English language learners understand informational text), writing communication (measures the lucidity of the information that English language learners provide to 
peers about story content), writing conventions (measures spelling, punctuation, capitalization, and usage), dictation (measures spelling, punctuation, and capitalization), and 
independent reading and library use, in which English language learners indicate the books and stories that they have chosen to read, and items borrowed from the library (as 
cited in Saunders, 1999). The assessments were administered in a small-group setting and scored by trained project staff and teachers under the direction of the author. The 
inter-rater reliability was 61 % based on exact agreement and 97% based on adjacent agreement (agreement within one point with a five-point scale). 
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Appendix A3.1 



Summary of study findings included in the rating for the reading achievement domain 1 









Authors’ findings from the study 














Mean outcome 
(standard deviation 2 ) 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students/ 
schools) 


IC&LL Comparison 

group group 


Mean difference 3 
(IC&LL- 
comparison) 


Statistical 
significance 5 
Effect size 4 (at a = 0.05) 


Improvement 

index 6 



Factual comprehension 


Grades 4-5 


Saunders & Goldenberg, 1999 (randomized controlled trial) 7 

32/1 14,88 12.12 2.76 

(3.03) (3.50) 


0.82 


Statistically 

significant 


+29 


Interpretive comprehension 


Grades 4-5 


32/1 


5.56 

(2.73) 


3.62 

(1.96) 


1.94 


0.80 


Statistically 

significant 


+29 


Average 8 for reading achievement (Saunders & Goldenberg, 1999) 








0.81 


Statistically 

significant 


+29 








Saunders, 1999 9 (quasi-experimental design) 10 








Performance assessment 


Grade 5 


84/10 


2.79 


2.25 


0.50 


0.94 


Statistically 


+33 








(0.65) 


(0.47) 






significant 




Comprehensive Test of Basic 


Grade 5 


84/10 


41.90 


34.51 


7.39 


0.63 


ns 


+24 


Skills (CTBS), Reading Subtest 






(10.64) 


(12.40) 










Average 8 for reading achievement (Saunders, 1999) 








0.79 


Statistically 


+28 
















significant 




Domain average 8 for reading achievement across all studies 








0.80 


Statistically 


+29 
















significant 





ns = not statistically significant 



1 . This appendix reports findings considered for the effectiveness rating and the average improvement index. Subgroup findings from the same studies are not included in these ratings, but are reported in Appendix A4.1 . 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are; a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 

3. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

4. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

5. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

6. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and the percentile rank of the average student in the comparison condition. The improvement index can 
take on values between -50 and +50, with positive numbers denoting favorable results. 

7. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the cluster- 
ing correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of Saunders & Goldenberg (1999), no correction 
for clustering was needed. The WWC did, however, correct for multiple comparisons. 

8. The WWC-computed average effect sizes for each study and for the domain across studies are simple averages rounded to two decimal places. The average improvement indices are calculated from the average effect size. 

9. Note that Instructional Conversations and Literature Logs were 2 of the 12 components of the language arts program that was implemented as part of the intervention. Therefore, findings were based on all 12 components, including 
Instructional Conversations and Literature Logs. 

10. In the case of Saunders (1999), corrections for clustering and multiple comparisons were needed, which changed the statistical significance of outcomes on the CTBS. 
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